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PREFACE 


This  report  reflects  the  results  of  an  effort  at  the  National  Aviation  Facili 
ties  Experimental  Center  to  test  and  evaluate  an  enroute  system  terrain- 
avoidance  function  in  conjunction  with  the  National  Airspace  System  Enroute 
Stage  A Model  3d2  system. 

The  operational  program  tape  used  throughout  testing  was  developed  for  the 
A3d2.1  system.  Discrepancies  recorded  were  limited  to  the  terrain-avoidance 
function  and/or  design  in  a simulated  low-altitude  environment.  Results,  con 
elusions,  and  recommendations  contained  in  this  report  are  operationally 
oriented  and  do  not  consider  program  size  and  processing  time  requirements  as 
a function  of  the  overall  system,  nor  do  they  consider  implementation- 
associated  factors. 
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INTRODUCTION 


PURPOSE. 

The  purpose  of  this  activity  was  to  test  and  evaluate  the  operational  suit- 
ability of  an  enroute  minimum  safe  altitude  warning  (E-MSAW)  function  utiliz- 
ing the  model  2d2.1  program  of  the  National  Airspace  System  (NAS).  This  report 
discusses  the  method  used  and  results  obtained  of  the  test  and  evaluation  of 
the  E-MSAW  function  conducted  in  the  System  Support  Facility  (SSF)  at  the 
National  Aviation  Facilities  Experimental  Center  (NAFEC) . 

BACKGROUND ■ 

The  E-MSAW  function,  which  provides  the  radar  controller  with  a displayed 
warning  of  a potential  collision  between  a tracked  aircraft  under  his  control 
and  terrain  and/or  ground  obstructions,  was  developed  and  built  into  the  NAS 
Model  A3d2.1  system,  also  containing  the  conflict  alert  (CA)  function.  This 
was  done  to  make  maximum  use  of,  and  to  be  consistent  with,  the  logic  of  CA. 

A series  of  program  validation  tests  of  the  3d2.1  prcgrr.m  with  E-MSAW  was 
conducted  in  the  SSF  at  NAFEC  by  ARD-140  personnel.  Based  on  the  favorable 
results  of  this  testing,  a series  of  operational  system  tests  of  the  E-MSAW 
function  was  scheduled  for  conduct  by  NAFEC,  ANA-110.  Th°  NAFEC  operational 
system  test  activity  was  conducted  between  December  11,  1975,  and  June  23,  1970. 


DISCUSSION 


GENERAL. 

Based  on  test  procedures  prepared  by  Federal  Aviation  Administration  (FAA) 
personnel  at  NAFEC,  a test  team  conducted  tests  ±n  the  SSF  to  evaluate  the 
performance  of  the  E-MSAW  function  in  a simulated,  low-altitude  environment, 
with  respect  to  detection  capability  and  adequacy  of  warning  provided. 

The  E-MSAW  function  is  designed  to  provide  early  detection  and  warning  of 
impending  violation  of  terrain  or  obstruction  clearance  criteria  by  tracked 
aircraft.  Altitude  data  for  mode  C and  non-mode  C tracked  aircraft  available 
from  conflict  alert  processing  are  utilized  to  predict  conflicts  between  an 
aircraft  and  the  ground/obstruction.  An  adapted  minimum  safe  altitude  (MSA), 
based  upon  the  terrain  and  obstructions,  is  compared  with  the  path  of  each 
track  projected  for  an  interval  of  time,  known  as  the  "search  time  for  terrain- 
avoidance  parameter"  (STTA) , and  a terrain  alert  is  generated  for  aircraft 
which  are  determined  to  be  too  low,  or  are  projected  to  become  too  low. 

Two  types  of  terrain  alerts  can  be  displayed  to  the  controller,  sort  box  alerts 
and  obstruction  alerts.  A sort  box  alert  indicates  that  an  aircraft  is  pro- 
jected during  STTA  to  be  within  a radar  sort  box  (RSB)  wtiose  MSA  is  higher 
than  the  current  or  projected  altitude  of  the  aircraft.  The  sort  box 
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FIGURE  1.  SORT  BOX  ALERT 


(16  nautical  miles  (nmi)  square)  is  outlined  on  the  controllers  plan  view 
display  (PVD) , and  the  MSA,  blinking,  in  large-size  characters,  is  displayed 
in  the  lower  left  corner  of  the  RSB.  A vector  line  representing  the  projected 
path  of  the  aircraft  during  STTA  is  displayed,  and  the  letters,  "GRND,"  appears 
in  field  E of  the  full  data  block  (figure  1) . 

An  obstruction  alert  indicates  that  an  aircraft  is  projected  during  STTA  to 
pass  too  close  to  an  obstruction  whose  MSA  is  higher  than  the  present  or  pro- 
jected altitude  of  the  aircraft.  An  obstruction  symbol  and  the  safe  altitude 
is  displayed  in  large-size,  blinking  characters  at  the  location  of  obstruction. 
As  in  the  sort  box  alert,  a vector  is  displayed,  and  GRND  is  displayed  in 
field  E of  the  full  data  block  (figure  2).  The  letters,  GRND,  will  not  replace 
"EMRG"  or  RFOF,"  which  mean  emergency  and  radio  failure,  respectively.  All 
alerts  detected  for  an  aircraft  during  STTA  are  displayed.  Therefore,  several 
sort  box/obstruction  alerts  can  be  displayed  simultaneously. 
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FIGURE  2.  OBSTRUCTION  ALERT 


For  all  test  sessions  at  NAFEC , parameter  STTA  was  kept  constant  at  150  seconds 
(2.5  minutes).  In  addition,  the  34  obstacles  adapted  for  testing  were  each 
assigned  a 5-nmi  radius  of  protected  airspace.  The  MSA  for  obstacles  varied 
from  6,000  feet,  as  the  lowest,  to  17,500  feet  as  the  highest.  The  MSA  for 
sort  box  adaptation  varied  from  1,300,  as  the  lowest,  to  17,000  as  the  highest. 
All  tests  involved  participation  of  NAFEC  controllers,  digital  simulation 
facility  (DSF)  personnel,  and  personnel  from  NAFEC  to  perform  observer,  data 
systems  specialist  (DSS) , and  system  engineer  (SE)  duties. 
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Thirty-eight  operational  system  tests  were  conducted  between  December  11,  1975, 
and  June  23,  1976.  During  this  time  frame,  five  versions  of  the  E-MSAW  functio 
identified  as  TAV001,  TAV002,  TAV003,  TAV004 , and  TAV005  were  tested.  Prior 
to  operationally  testing  each  version,  a minimum  of  two  shakedown  tests  were 
conducted  to  verify  stability  of  the  new  version.  The  appendix  contains  a 
summary  of  all  test  sessions  (shakedown  and  operational)  conducted. 

ENVIRONMENT. 

All  test  sessions  were  conducted  in  the  SSF,  a laboratory  model  developed  at 
NAFEC  for  enroute  system  testing  (figure  3).  The  SSF  is  divided  into  two 
12-sector  laboratories,  each  one  differing  only  in  equipment  used  to  drive  the 
displays.  These  include  the  computer  display  channel  (CDC)  and  the  9020E  dis- 
play channel  complex  (DCC) . Tests  could  be  and  were  conducted  using  either 
configuration . 

Each  version  of  the  E-MSAW  function  was  built  into  the  CA2111  system,  which  is 
an  A3d2.1  A3DL055  NAS  system  that  includes  version  11  of  conflict  alert.  The 
addition  of  the  E-MSAW  function  to  that  system  did  not  change  the  functional 
characteristics  of  conflict  alert  in  any  way.  For  the  CDC  laboratory,  CDC 
version  38F8  was  used,  and  for  the  DCC  laboratory,  the  DCC  version  507Y  was 
utilized . 

Sector  configuration  was  identical  in  both  laboratories  and  was  based  upon  a 
hypothetical  air  route  traffic  control  center  (ARTCC)  geography  known  as  the 
universal  data  set  (UDS).  Of  the  12  sectors  available,  the  7 low-altitude 
enroute  sectors  (sectors  1,  2,  3,  4,  5/6,  7/8/9,  and  10/11)  were  utilized  for 
test  sessions. 

Simulated  radar  inputs  to  the  SSF  were  provided  by  the  DSF  for  6 of  the  .12 
radar  sites  adapted  for  the  UDS. 

One  traffic  sample  was  utilized  during  testing.  Air  traffic  flow,  density, 
types  of  aircraft,  and  equipment  were  relatively  realistic  to  that  encountered 
in  a live  environment.  The  sample  was  a mix  of  enroute,  departures  and  arrival 
and  provided  a 2-hour  period  of  air  traffic  control  situations.  In  addition, 
preplanned  terrain/obstacle  conflicts  and  aircraf t/aircraf t conflicts  were 
programmed  into  the  traffic  sample.  All  traffic  movement  was  limited  to  the 
low-altitude  airspace  (flight  level  230  and  below). 

METHODOLOGY 


GENERAL. 


To  test  and  evaluate  the  E-MSAW  function  with  respect  to  detection  capability 
and  adequacy  of  warning  provided,  five  major  areas  of  evaluation  were  selected 
and  remained  constant  throughout  testing.  The  areas  of  evaluation  included 
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SUPERVISORY  A.  #2 


FIGURE  3.  SYSTEM  SUPPORT  FACILITY  LABORATORY  CONFIGURATION 


(1)  overall  performance  of  detection/alert,  (2)  timeliness  of  the  alert,  (3) 
false  alert  occurrences,  (4)  attention-getting  suitability,  and  (5)  quality 
of  the  displayed  information  and  alert. 

The  DSF,  according  to  a preplanned  traffic  sample,  was  used  to  generate  target 
movement  on  specific  routes  with  associated  flight  plans  (taped  inputs) . 
Approximately  228  aircraft  were  tracked  during  each  2-hour  test  period.  Each 
test  period  was  comprised  of  operational  air  traffic  control  situations  in  a 
sterile  test  environment;  i.e.,  no  code  garbling,  no  radar  noise,  no  jitter. 

Throughout  testing,  controllers  and  observers  were  asked  to  record  any  dis- 
crepancies in  the  performance  of  the  E-MSAW  function,  the  A3d2.1  program  with 
E-MSAW,  or  the  conflict  alert  function  working  in  combination  with  the  A3d2.1/ 
E-MSAW  function.  Observers  were  also  asked  to  record  false  alerts,  alerts 
generated  too  soon,  and/or  alerts  generated  too  late. 

During  test  sessions,  specific  sectors  were  selected  to  test  certain  situations 
that  were  not  included  in  the  traffic  sample.  These  situations  included  descent 
below  assigned  altitude,  turns  into  and  away  from  unsafe  areas,  multiple  air- 
craft in  the  same  sort  box,  multiple  aircraft  toward  the  same  obstacle, 
descents  on  obstacles,  climbs  into  obstacles,  priorities  of  codes  7700 
and  7600  over  GRND  in  data  blocks,  use  of  the  suppression  logic,  and  turns 
toward  a sort  box  already  displayed  for  another  aircraft.  In  addition  to 
these  situations,  which  were  repeated  numerous  times,  all  input/output 
typewriter  (IOT)  messages  associated  with  the  E-MSAW  function  were  exercised 
at  various  times  during  test  sessions. 

As  the  E-MSAW  implementation  (into  the  NAS  system)  was  to  make  maximum  use  of 
and  to  be  consistent  with  the  logic  of  conflict  alert,  situations  were  devel- 
oped to  determine  if  any  problems  existed  when  both  functions  were  activated 
for  the  same  aircraft.  Controllers  and  observers  were  asked  to  record  any 
degradation  of  either  function  during  saturation-type  tests. 

DATA  COLLECTION. 

Manually  collected  data  recorded  by  observers,  controllers,  and  debriefing 
sessions  were  used  as  the  basis  for  analysis  of  test  results.  Terrain  alert 
printouts,  conflict  alert  printouts,  and  automatic  data  collected  on  the 
system  analysis  recordings  (SAR)  were  used  in  conjunction  with  the  manually 
collected  data. 


RESULTS 


Results  were  based  on  (1)  debriefing  sessions  that  followed  each  test  session, 
(2)  test  observer  logs,  (3)  analysis  of  online  printouts,  and  (4)  scrutiny  of 
the  data  reduction  and  analysis  printouts  obtained  from  the  automatic  data 
collected  on  the  system  analysis  recordings.  The  major  emphasis  in  conducting 
E-MSAW  testing  was  placed  on  assessing  the  performance  of  the  A3d2.1  E-MSAW 
function  with  respect  to  detection  capability  and  adequacy  of  warning  provided. 
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Five  versions  (TAV001  through  TAV005)  of  the  E-MSAW  function  were  evaluated 
during  this  test  effort.  Although  controller  response  to  the  E-MSAW  concept 
was  favorable  throughout  testing,  the  first  three  versions  were  considered  to 
be  unsatisfactory  from  an  operational  standpoint.  This  unsatisfactory  condition 
was  caused,  in  part,  by  the  high  rate  of  "false  alarms"  being  generated  during 
testing.  A total  of  39  program  trouble  reports  (PTR's)  were  written  against 
the  first  three  E-MSAW  versions.  The  majority  of  the  PTR's  concerned  false 
alerts.  Additional  problems  encountered  with  TAV001 , TAV002 , and  TAV003 
included  overriding  EMRG  and  RDOF  priorities,  failure  to  detect  and  generate 
an  alert  (isolated  cases),  and  erratic  display  of  information  in  field  E of 
the  full  data  block. 

With  release  of  TAV004  and  TAV005  came  a more  stable  E-MSAW  program  and  cor- 
rections to  problem  areas  encountered  in  previous  versions.  Although  the 
latter  two  releases  were  not  error  free,  controllers  agreed  that  the  problems 
encountered  did  not  derogate  the  desirability  of  having  the  E-MSAW  function 
as  a part  of  the  enroute  NAS  program. 

The  timing  of  the  appearance  of  the  displayed  information  (sort  box/obstacle, 
flashing  altitude,  and  GRND)  was  considered  adequate  for  most  situations; 
i.e.,  the  alert  was  displayed  in  time  for  the  controller  to  alert  the  pilot 
and  for  safe  response. 

With  some  minor  exceptions,  the  performance  of  the  E-MSAW  function  was  found 
to  be  adequate  for  enroute  aircraft.  However,  controllers  agreed  that  major 
problems  (false  alerts  and  clutter)  could  exist  when , (1)  aircraft  were 
departing  from  or  arriving  at  airports  having  no  approach  control  service  or 
automated  radar  terminal  system  (ARTS)  service,  (2)  track  control  is  retained 
by  the  ARTCC,  and  (3)  these  airports  are  not  precisely  defined  as  terminal 
areas  in  adaptation.  A functional  change  for  TAV005  was  designed  to  prevent 
this  problem;  however,  due  to  time  limitations,  the  change  was  not  subjected 
to  detailed  testing  for  verification. 

Additional  problems  with  arriving  aircraft  occurred  when  the  airport  of  intended 
landing  was  between  the  aircraft  position  and  an  obstacle  beyond  the  airport, 
but  within  STTA  minutes  of  flying  time.  The  STTA  projection  beyond  the  air- 
port generated  an  obstruction  alert  which  the  controller  considered  false  and 
distracting.  Again,  a functional  change  to  prevent  this  type  of  problem 
could  not  be  tested  due  to  time  limitations. 

The  STTA  projection  can  also  cause  alerts  to  be  generated  for  aircraft  cleared 
to  hold  at  a fix  which  has  terrain  and/or  an  obstruction  with  a higher  MSA  than 
the  aircraft  altitude  beyond  it  and  the  aircraft  has  not  yet  reached  the  point 
where  the  track  is  dropped. 

In  addition  to  the  possible  false  alerts  generated  by  arriving/departing  air- 
craft, false  alerts  and  late  alerts  can  be  generated  by  enroute  aircraft.  The 
majority  of  these  false/late  alerts  can  be  attributed  to  the  lag  in  the  terrain- 
avoidance  vector  line.  To  compound  the  problem,  the  terrain-avoidance  vector 
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Throughout  testing,  all  E-MSAW-associated  inputs  were  exercised.  The  inputs 
were  made  dynamically  from  the  computer  entry  device  (CED) , the  data  entry 
control  (DEC),  and  the  IOT.  No  inputs  were  exercised  from  the  card  reader. 

All  output  for  the  associated  input  was  as  specified. 

During  several  test  sessions,  short  periods  of  time  were  devoted  to  terrain 
alert  storage  saturation.  In  conjunction  with  the  terrain  alert  saturation, 
controllers  vectored  aircraft  into  conflict  to  determine  the  impact,  if  any, 
that  this  saturation  would  have  on  the  conflict  alert  program.  There  were  no 
reports  or  observations  of  degradation  of  either  the  NAS  system  or  the  conflict 
alert  function  once  terrain  alert  storage  saturation  was  attained  (indicated 
by  the  IOT  output  message,  "WARNING — TERRAIN  ALERT  STORAGE  SATURATED"). 
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Throughout  testing,  all  E-MSAW-associated  inputs  were  exercised.  The  inputs 
were  made  dynamically  from  the  computer  entry  device  (CED) , the  data  entry 
control  (DEC),  and  the  IOT.  No  inputs  were  exercised  from  the  card  reader. 

All  output  for  the  associated  input  was  as  specified. 

During  several  test  sessions,  short  periods  of  time  were  devoted  to  terrain 
alert  storage  saturation.  In  conjunction  with  the  terrain  alert  saturation, 
controllers  vectored  aircraft  into  conflict  to  determine  the  impact,  if  any, 
that  this  saturation  would  have  on  the  conflict  alert  program.  There  were  no 
reports  or  observations  of  degradation  of  either  the  NAS  system  or  the  conflict 
alert  function  once  terrain  alert  storage  saturation  was  attained  (indicated 
by  the  IOT  output  message,  "WARNING — TERRAIN  ALERT  STORAGE  SATURATED"). 


CONCLUSIONS 


From  analysis  of  all  the  data  collected,  it  is  concluded  that: 

1.  With  respect  to  detection  capability  and  adequacy  of  warning  provided, 
although  the  E-HSAW  function  performed  adequately  for  most  enroute  situations, 
late  or  false  alerts  can  be  expected  due  to  the  lag  between  the  terrain 
avoidance  vector  line  and  the  true  aircraft  heading. 

2.  For  arriving  and  departing  aircraft,  false  alerts  and  an  increase  in 
clutter  can  be  expected  unless  non-approach-control-serviced  airports  are 
precisely  defined  as  terminal  areas  in  facility  adaptation. 

3.  Inclusion  of  the  E-MSAW  function  into  the  A3d2.1  conflict  alert  system 
did  not  degrade  the  performance  of  the  CA  function. 

4.  In  certain  instances,  displayed  information  can  be  difficult  to  interpret 
or  can  be  misinterpreted. 

5.  Discrepancies  between  information  displayed  to  the  controller  and  infor- 
mation printed  out  on  the  HSP  during  the  generation  of  terrain  alerts  can 
occur . 


RECOMMENDATIONS 


Predicated  on  the  results  and  the  conclusions  derived  from  these  tests  of  the 
E-MSAW  function,  the  following  recommendations  are  presented: 

1.  Conduct  further  E-MSAW  testing  at  NAFEC  to  optimize  display  information, 
parameter  values,  and  verify  functional  changes  to  the  program  currently 
available  or  as  added  in  the  future. 

2.  Refine  the  E-MSAW  program  design  so  that  it  makes  use  of  flight  plan 
information  to  reduce  false  or  late  alerts  for  aircraft  nearing  an  expected 
turn . 

3.  Improve  the  track  azimuth  and  track  position  prediction  process  to  more 
closely  reflect  true  aircraft  heading  in  order  to  reduce  false  or  late  alerts 
for  vectored  aircraft. 

4.  Following  the  above,  conduct  a period  of  field  evaluation  tests  of  the 
E-MSAW  function,  leading  to  possible  implementation  into  the  enroute  NAS. 
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